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Abstract—The development of manufacturing requires in- 
telligent solutions like digitization, real-time monitoring, and 
simulation techniques through appropriate digital manufactur- 
ing tools. This paper presents the design development of a 
Digital Twin for a robotic stacking workcell, working prin- 
ciple design, and trajectory optimization. A new programming 
decision was implemented concerning the different types of 
workpieces through ABB integrated vision to recognize the 
types and offset angles of workpieces. This work also involves 
creating a novel web platform for remote monitoring of the 
robotic cell. This platform also can send alerts in case of 
emergency events and provide control access of ABB robotic 
arm to authorized engineers with the help of RWS, a URL- 
based command, which cooperate with C# based clients. The 
goal of this research is to create a workcell-based model for 
intelligent robotic stacking that incorporates critical concepts 
such as Cyber-Physical Systems, digitization, data acquisition, 
continuous monitoring, and intelligent decision in an innovative 
manner. 


I. INTRODUCTION 


In the era of Industry 4.0, manufacturing enterprises are 
under great pressure to reduce costs and respond rapidly to 
the changing marketing environment, such as shorter product 
lifecycle, stochastic orders, individualized product, and the 
unplanned disruptive manufacturing process [1] [2]. To meet 
these challenges, the rapid integration of manufacturing 
resources and real-time decision-making at change scales 
on a shop floor and even globally has become an urgent 
issue. For these influences, many efforts have been devoted 
to developing cyber-physical production systems (CPPS) [3]. 
The most typical form of CPPS is the digital twin system. 
[4] 

The digital twin is defined as “‘a formal digital representa- 
tion of some asset, process or system that captures attributes 
and behaviors of that entity suitable for communication, 
storage, interpretation or processing within a certain context” 
by the industrial internet consortium (IIC) [5] [6]. Some other 
scholars have given their interpretations in terms of digital 
twins, yet there is no common definition of this concept, nor 
the methodology, standards, or norms regarding its imple- 
mentation [7]. Nevertheless, it is believed that all digital twin 
systems should have some common characteristics in that it 
iS a two-way mapping of a physical entity (i.e. the physical 
equipment in the factory shop floor) and a corresponding 
virtual model (created in computers) [5]. 


Fig. 1: The Digital Twin of a robotic stacking workcell 
overview (A: Cognex InSight Camera; B: Recognition-and- 
Grab Area; C: Checkboard) 
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Fig. 2: Tool Center points: (a): Robot tool mounting point; 
(b): Gripper mounting point; (c): Magnetic sucker 
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Digital twin technology, as an important means to promote 
the digital transformation of enterprises and the development 
of the digital economy, has established a universal theoretical 
and technical system [8]. However, it is short of practical 
applications in industrial automation and intelligent manu- 
facturing. Therefore, the practice on implementing the digital 
twin in real industrial applications forms the subject matter 
of this paper. 

This paper covers the development of a digital twin 
system for a small-scale robot manufacturing station. As 
a first attempt, we develop the system based on a simple 
task in manufacturing applications, i.e. robot grasping. Our 
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Fig. 3: (a): Target stacking area with multiply coefficients and three kinds of workpieces; (b): Program operation sequence; 


(c): The program loaded and shown on the robot flex pendant 
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Fig. 4: 


proposed digital twin system incorporates tool installation 
procedures, complete workstation modeling, design working 
process, and the optimization of stacking trajectory planning. 
It endeavors to achieve a more efficient visualization-based 
stacking system by combining C#tutoriala-based client with 
Unity 3D and Vue-based web design, utilizing Robot Web 
Service (RWS) — a combination adhering to Rest API and 
HTTP protocols [9] [10]. 


Il. THE DIGITAL TWIN OF A STACKING STATION 


The physical characteristics of workcell components are 
modeled through modern blender tools and_ visualized 
through Unity 3D. The DT representation of the virtual 
modeling workcell is illustrated in Fig.1, performing com- 
prehensive control over the manufacturing process, encom- 
passing the design, planning, testing, collision prevention, 
fault detection, and monitoring. 

In Fig.1.A, the ABB’s integrated vision system enables 
workpiece type recognition and offset angle calculation. The 
workpieces are transported using a conveyor. The centerpiece 
of the setup is the 6 DoF articulated arm robot, ABB IRB120 
[11], equipped with a gripper that allows seamless tool 
changes, including a magnetic sucker tool used to pick up 


Cell signals configuration in the robot controller 


and place workpieces on the checkboard. Moreover, the CAD 
design includes supplementary tools such as grinding and 
polishing tools.” 

In Fig.2, three different tool center points (TCP) were 
displayed for: (a): the robot’s tool mounting point, (b): the 
gripper mounting point, and (c): the magnetic sucker tool, 
which is used to grab the iron workpiece. 


III. WORKING PROCESS OF THE WORKCELL 
A. Robot programming design with IO signals configuration 


Before creating the Digital Twin, ABB’s product, Robot- 
Studio was used to present the robotic workcell, generate 
collision-free tool paths, and program the robot [11]. The 
software also considers sensor data, analysis of motion 
results, and evaluation of robot energy consumption. 

Events have been established using I/O signals that serve 
as communication channels between the controller and exter- 
nal equipment. For instance in Fig.4, DO_02 signal enables 
activating the conveyor. 

The programming included robot movements for tool 
changes and picking up and placing workpieces between the 
gravitational aligning table, fixture system, collision checks, 
and outfeed conveyor. 


The robotic stacking sequence is illustrated in Fig6 to 
certify all workpieces are entirely packed. There are three 
working stages: 


e In the Ist stage, the workpieces are sent through the 
conveyor and recognized by the Cognex Insight Cam- 
era, then stop immediately when the infrared distance 
sensor at the end of the conveyor is triggered to ensure 
a relatively precise position and orientation before it 
is packed. However, small position errors could be 
compensated depending on the offset angle given by 
integrated vision, which will be discussed later. 

e Moving on to the 2nd stage, inspired by the concept 
introduced by [12], the workpiece is brought to the 
designated checkboard depicted in Fig.3. Here, the 
optimal orientation for stacking is calculated. Initially, 
a 3 x 3 x 3 occupation-record field is established to 
represent the stacking area. Subsequently, the robot 
assigns a score to each potential stacking point based 
on the types of workpieces. Subsequently, the robot 
multiplies three corresponding coefficients along the X, 
Y, and Z axes according to the desired stacking priority 
to identify the best area for stacking. For example, 
Fig.3.(a) illustrates the scoring mechanism along the Z- 
axis, generating three different scores while evaluating 
the ideal position for stacking the square workpiece. 

e In the 3rd stage, the robot waits for a new workpiece 
transported by the conveyor or retrieves the magnetic 
sucker back and goes safety point. 


Similar to [13], sensor signals were configured in the base 
of workstation logic and synchronized to the virtual robot 
controller IO environment shown in Fig.4. 


After designing all robot actions the virtual representation 
could show programs, each target point, orientation, pro- 
gramming sequence, and communication within the upper 
computer. The robot stacking sequence, communication pro- 
grams, and robot movements are illustrated in Fig.3.(b).(c). 

Finally, the well-integrated data could be sent through 
RWS (Robot Web Service), including Robot joint data, DO 
signals, and production information to the back-end client, 
which will be explained later. 
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Fig. 6: Comprehensive representation of the stacking work- 
flow 


B. Vision Recognition and Offset Angles 


Due to the uncertainty in the types and angles of 
workpieces transported by the conveyor, this study fo- 
cuses on workpiece type and offset angle recognition. In 
Fig.5.(a).(b).(c), three types of workpieces are initially iden- 
tified. Then, an automated process calculates the signifi- 
cant offset angle, which is represented as a green cross- 
hair cursor. For instance, in Fig.5.(b), when the rectangular 
workpiece is not aligned straight with the conveyor, the offset 
angle can be calculated and subsequently incorporated into 
the robot’s >*MOVEL” command using the ’Offs’ function 
while attempting to grasp the workpiece. As a result, the 
robot can effectively optimize its routines and achieve central 
symmetry positioning for the workpiece. 


IV. THE OPTIMIZATION OF ROBOT STACKING 
TRAJECTORY PLANNING 


To optimize the energy consumption of the entire process- 
ing procedure, four steps are designed as follows: 


e MoveJ to the prepared grab position. 

e MoveL to grab the workpiece. 

e MoveL to the safety point above the stacking area. 
e MoveL to place the workpiece. 


Bin Packing Total Energy and Power Measurement 


Energy Consumption(J) 


Robot Power(w) 


Energy Consumption{[J| 


0 10 20 = 30 A0) 10) 


Robot Power|w]| 


60 70 80 90 00 110 


Tool change Packing rectangle Packing square Robot Video Packing circle 
workpiece workpiece Inspection workpiece 
Time[s] 


Fig. 7: Records of the total energy (marked in red) and total power (marked in blue) of stacking sequence processing three 
kinds of workpieces (rectangle, square, circle) with tool change and robot video inspection 
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Fig. 8: Web-based robotic stacking system platform for monitoring and remote control 


To ensure minimum energy and time consumption, MoveJ 
manipulates all joints together, enabling the robot to move 
in the fastest and unique routine, albeit with partial lack of 
controllability [14]. However, the reliability and stability of 
transporting the workpiece are also crucial. Therefore, the 
workcell is designed to use ”MoveL” for linear workpiece 
transportation, following a straight line path between the 
current position and the destination [14]. 


The overall performance of the workcell is monitored and 
recorded in Fig.7. This monitoring includes capturing the 
real-time Power and Total Energy curve during typical oper- 


ating conditions. These conditions involve stacking each type 
of workpiece (rectangle; square; circle), tool change, and 
robot video inspection. The optimized results of the designed 
optimized stacking trajectory planning can be observed as 
follows: 


e The smoothness of the energy curve shown in the red 
curve reflects the robot’s ability to maintain a steady 
energy consumption during the stacking process. This 
indicates that it avoids unnecessary energy waste and 
follows a safe routine for transportation. 

e Secondly, the power curve consistently stays within 
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Fig. 9: Initialize panel (marked in blue) and 3D model of the workplace (marked in red) and live-stream of 3D model of the 
workplace and working efficiency of the stacking (marked in black) and the control panel of IO settings (marked in yellow) 


the designated orange box region, maintaining a low 
level with minimal power peaks. This allows the robot 
to avoid abrupt acceleration or deceleration during the 
stacking process. As a result, the robot can efficiently 
complete the stacking task while simultaneously re- 
ducing peak energy consumption and minimizing the 
overall system load. 


V. WEB BASED PLATFORM FOR ROBOTIC STACKING 
REMOTE CONTROL AND MONITORING 


This project aims to implement a web-based monitoring 
and remote control system to keep pace with Industry 4.0. 
The system utilizes Web services with the RWS (Robot Web 
Services) [9]protocol to acquire real-time information from 
the actual workplace. RWS is an HTTP-based protocol ca- 
pable of retrieving current data through URLs and receiving 
data in XML or JSON formats. Additionally, the platform 
allows users to remotely control robotic arms by sending 
desired locations for precise positioning. The received data 
is processed and stored in two databases: the first database 
stores production logs together with error logs. In contrast, 
the second database stores the current IO signals value, 
displaying the machine status. 

The working process of the web-based platform is illus- 
trated in Figure 8. For data acquisition, as shown on the right, 
the C# [15]-based client automatically sends requests based 
on RWS [9] every | second in JSON format. After receiving 
the data, the client processes and stores it in the local 
database, and it collaborates with Unity, where the model 
moves according to the received six-axis data. Moreover, in 
case of critical events, such as axis angles exceeding the 
safe range or the gripper getting stuck, the platform alerts 
engineers online to ensure the robot’s safety. 

For remote control, engineers can manipulate the model by 


dragging buttons to adjust the six axes. The data is then sent 
to the client, which forwards the URL to update the angles. 
This results in synchronized movement of both the virtual 
and actual models. As exhibited in Fig.9, the following will 
represent a few functions that the web-based platform can 
represent. 


e Func.1l: The Ith section blue section.Robot and Con- 
troller Selection. Machine Control. The platform allows 
users to control the entire machine’s power, offering 
remote start, pulse, and stop functions. This feature 
ensures precise control over production processes for 
improved efficiency. 

e Func.2, the 2nd orange section, modeled Workplace 
Visualization. The platform leverages Unity to provide 
users with a 3D modeled representation of their work- 
place. This visualization aids in better understanding 
the layout and arrangement of robotic systems within 
the factory floor. 

e Func.3, the 3rd green section.Robot and Controller Se- 
lection. Real-Time Workplace Status. Users can monitor 
the real-time status of the workplace through the plat- 
form. This includes information on robot movements, 
production progress, and error notifications, enabling 
swift intervention when required. 

e Func.4: The 4th section.Robot and Controller Selection. 
Efficiency Analysis and Charts. Data collected via RWS 
[9]is used to generate efficiency charts, providing valu- 
able insights into production performance. Users can 
analyze these charts to identify bottlenecks, optimize 
processes, and enhance overall productivity. 

e Func.5: The 5th yellow section.Robot and Controller 
Selection. DO System Management. Users can control 
the entire DO system, regulating the activation and de- 
activation of various components such as conveyors and 


electromagnets. This allows for dynamic adjustments to 
the production line configuration. 


VI. REMOTE MONITORING AND CONTROL 


This project presents a web-based remote control system 
for an ABB robotic arm, illustrated in Fig.8, integrating 
the RWS (Robotic Web Services) [9]protocol and RAPID 
language. The system offers two types of control: local 
control and remote control. When operating in remote control 
mode, the robotic arm’s RAPID program detects user com- 
mands from a C#tutoriala-based client and creates a variable 
called ’location-R.” The client will send the actual location 
of the targeted position by using a specific format. This 
variable enables the robotic arm to be precisely controlled 
by the user, either by dragging sliders or using the control 
panel to manipulate the six-axis degree. For the dragging 
method, one drag will cause the J1 to J6 to move a specific 
degree, and the data will be received in the client and 
processed. After that, it will be sent to the actual ABB 
robotic arm as formulated data. In addition, this platform can 
also open/close the electromagnet, open/close the conveyor, 
open/close the alarm, and open/close the gripper. These can 
be realized by changing the value of IO systems’ values. 
RWS [9] provides the URL that can monitor and change the 
value of the DO signals, making remote control a realized 
dream. 


VII. CONCLUSION 


This paper has successfully developed a Digital Twin for 
a robotic stacking workcell. Our contribution includes the 
following aspects: 


e Through blender tools and visualizes it through Unity 
3D, construct the digital twin robotic stacking workcell 
and remote monitoring and control platform communi- 
cating by RWS between Vue based web platform and 
the c# based client. 

e Employed robot programming through tool assembly 
and IO signals configuration based on TCP(Tool center 
points) and working principles. 

e Implemented a new working process concerning the 
different types of workpieces through ABB integrated 
vision to recognize the types and offset angles while 
minimizing energy consumption and ensuring reliability 
through monitoring energy and power curve. 
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